Search for: All records

This paper addresses the problem of identifying whether/how a black-box autonomous system has regressed in performance when compared to previous versions. The approach analyzes performance datasets (typically gathered through simulation-based testing) and automatically extracts test parameter clusters of predicted performance regression. First, surrogate modeling with quantile random forests is used to predict regions of performance regression with high confidence. The predicted regression landscape is then clustered in both the output space and input space to produce groupings of test conditions ranked by performance regression severity. This approach is analyzed using randomized test functions as well as through a case study to detect performance regression in autonomous surface vessel software. 

This paper offers a multi-layer planning approachfor autonomous surface vessels (ASVs) that must adhere togood seamanship practices and the International Regulationsfor Prevention of Collisions at Sea (COLREGS) [1]. The ap-proach combines novel situational awareness logic with motionprimitive-based planners in a receding horizon framework.Further, ship domain and ship arena concepts are used todevelop risk metrics that capture COLREGS compliance andthe notion of good seamanship. By relying on metrics-drivenmotion planning as opposed to rule-based conditions, theproposed framework scales naturally to non-trivial single-vessel and multi-vessel situations. The planner is evaluatedusing adaptive, simulation-based testing to statistically comparethe performance to other standard methods. Finally, proof-of-concept field experiments are presented on a subscale platform 

The current state-of-the-art for testing and evaluation of autonomous surface vehicle (ASV) decision-making is currently limited to one-versus-one vessel interactions by determining compliance with the International Regulations for Prevention of Collisions at Sea, referred to as COLREGS. Strict measurement of COLREGS compliance, however, loses value in multi-vessel encounters, as there can be conflicting rules which make determining compliance extremely subjective. This work proposes several performance metrics to evaluate ASV decision-making based on the concept of "good seamanship," a practice which generalizes to multi-vessel encounters. Methodology for quantifying good seamanship is presented based on the criteria of reducing the overall collision risk of the situation and taking early, appropriate actions. Case study simulation results are presented to showcase the seamanship performance criteria against different ASV planning strategies. 

Abstract This paper explores the use of autonomous underwater vehicles (AUVs) equipped with sensors to construct water quality models to aid in the assessment of important environmental hazards, for instance related to point‐source pollutants or localized hypoxic regions. Our focus is on problems requiring the autonomous discovery and dense sampling of critical areas of interest in real‐time, for which standard (e.g., grid‐based) strategies are not practical due to AUV power and computing constraints that limit mission duration. To this end, we consider adaptive sampling strategies on Gaussian process (GP) stochastic models of the measured scalar field to focus sampling on the most promising and informative regions. Specifically, this study employs the GP upper confidence bound as the optimization criteria to adaptively plan sampling paths that balance a trade‐off between exploration and exploitation. Two informative path planning algorithms based on (i) branch‐and‐bound techniques and (ii) cross‐entropy optimization are presented for choosing future sampling locations while considering the motion constraints of the sampling platform. The effectiveness of the proposed methods are explored in simulated scalar fields for identifying multiple regions of interest within a three‐dimensional environment. Field experiments with an AUV using both virtual measurements on a known scalar field and in situ dissolved oxygen measurements for studying hypoxic zones validate the approach's capability to quickly explore the given area, and then subsequently increase the sampling density around regions of interest without sacrificing model fidelity of the full sampling area.